An algorithm for computing the capacity of arbitrary discrete memoryless channels

where here the probability is understood to range over the random choice of code as well as the selection of (r,s). (64) il i=l We find the expected value (over e and s) It follows that d(y,x(r))-+ c@ + 2p in probability and therefore Pr{E,} + 0 as n + co. We are left with the evaluation of Pr{E*}. We write Pr{E,} 5 Pr{d(x(r)& I n(aj + 6p + E), for some r # 1 Ix(l) transmitted} 5 2nRr* Pr{d(x(2)& 5 n(@ + &p + E)}. (67) But d(XW,Y,) = wt(x(2) 0 x(1) 0 z(s) 0 e), (68) where it denotes the number of l's in the binary n-tuple, and x(2) and x(l) are independent Bernoulli n-sequences with parameter +. Thus, for any E > 0, Pr{E2} 5 2 nR122 " (H(.p+Pp)+O(lnn,n)+s,,2-n (69) where 2n(H(olii+Pl)+o(*nn'n)+c') denotes the number of points in the decoding sphere centered at y2. Consequently, if , RI2 < 1-H(aj + up)-E, (70) then Pr{E,} + 0, as 12-+ co. Collecting the constraints of (60) and (70), we see that if R2 = RI2 < 1-H(ap + ccp) (71) then E{Bl(")(e) + jjz(" '(e)} = E{jl(")(e)} + E{&(")(e)} + 0. (72) Since the best code behaves better than the average, there must exist a sequence of [(2nRr,2 " R~,2 " R12), n] codes for n = 1,2,.. . , with RI = C(aj + ~?p) + H(a)-E Rz = C(aj + c7p)-E such that &(")(e) + j&(")(e)-+ 0, and thus jl(")(e) + 0, bz(")(e) + 0. Taking the limit of (R1,R2) as E-+ 0 proves the theorem. Abstract-A systematic and iterative method of computing the capacity circumstances, it is exponentially decreasing. Finally, a few inequalities of arbitrary discrete memoryless channels is presented. The algorithm is that give upper and lower hounds on the capacity are derived. very simple and involves only logarithms and exponentials in addition to elementary arithmetical operations. It has also the property of mono